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ABSTRACT 


This thesis presents an analvsis of the application of the Modular Command and 
Control Evaluation Structure (}{CES) to the Identification Friend, Foe, or Neutral 
(IFFN) Joint Testbed. The MCES and IFFN Testbed evaluation approaches are also 
compared. \{CES 1s a structured approach to evaluate Command and Control (C2) 
svstems which uses standard and evolving operational research tools. The MCES 
approach provided the IFFN Joint Testbed with an air defense C2 system architecture 
which became a descriptive tool for C2 analysts to define and evaluate measures to 
determine the effectiveness of competing air defense C2 systems. This IFFN 
application served as an evaluation and refinement of MCES as well as a tool for 
assisting the IFFN Joint Test Force in evaluating U.S. air defense C2 systems in the 
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I. INTRODUCTION 


A. NATURE OF THE PROBLEM 

How much of a force multiplier can be attributed to a particular command and 
control (C-) systems? Given several alternative ce svstems, Which is best? What has 
to be measured to determine this difference? Are all the relevant factors taken into 
account? These are complex c? questions being asked by the Joint Chiefs of Staff as 
well as other senior military commanders as thev are faced with acquiring, testing, and 
operating (on svstenis. 

A methodology is needed to describe the c? systems architecture which will allow 
analysts to measure (on systems response and attribute the effectiveness of that 
response to the elements or structural relationships which form that c? system. | [iene 
is a definite need for generic tools to evaluate ce systems and architectures. What was 
lacking in current C? evaluation methodologies was a method to relate Cc? Svstems to 
measures of its contribution to force effectiveness and mission accomplishment. In the 
past, C* evaluation has been conducted in a piecemeal fashion with assorted evaluation 
tools only for specific parts of the problem. The Joint Chiefs of Staff Command, 
Control and Conimmunications Systems Directorate’s (JCS C3S) recognized these needs 
and required development of a paradigm to evaluate competing C* architectures 
(Ref. l: p. 8]. The Modular Command and Control Evaluation Structure (\{CES) 


attempts to address this need. 


B. NCES METHODOLOGY 

VICES was developed as a structured approach to evaluate om svstems and uses 
standard and evolving operations research tools. MCES attempts to integrate the 
previous efforts of C* users and analytic organizations to form a single C* evaluation 
package. 

The MCES is composed of seven separate modules which guide analvsts through 
the command and control evaluation. Figure 1.1 represents the seven modules of the 
MCES methodology and the output from each module. The first module is used by 
the analvst and operational user to define the particular ce problem. The nexteeines 
modules set the terminology and theorv to describe the c? system architecture which 


; ) ; 
permits analysts to model the C~ system and its operation. Inherent in the 
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methodology is a need to describe the C? system as an architecture integrating physical 
elements and process functions into a structural framework. {CES shares common 
terminology of current Cc systems evaluation methodologies. The MCES defines 
“architecture” as a description of an integrated set of systems Whose physical entities, 
Structure and functions are coherently related. The architecture provides a 
representation Which will eventually lead to the ability to measure the Co system 
response and the effectiveness of directing forces to accomplish the mission. The on 
theory behind these modules is robust enough to allow analysts to reconfigure the 
particular C2 system physically or structurally within the architecture during the C? 
evaluation. In module 5, measures are developed which will be used to discriminate 


° ~ « ™ ° Pd . » 
between alternative configurations of the architecture. When the measures for the C* 


svstem have been identified, the sixth module requires that a suitable data generator be 
selected or developed to derive the values for the measures selected. The final ICES 
module is used to aggregate and evaluate the C* measure results in order to determine 


the optimal Cc? svstem for a particular mission. [Ref. 1: pp. 10-23] 


Cc. MCES EVOLUTION 

The Modular Command and Control Evaluation Structure (MCES) was 
developed at the Naval Postgraduate School with research support from the Military 
Operations Research Society (MORS) and monetary support from different military 
agencies [Ref. 2: p. 13]. The MCES development started with military community 
research and discussion concerning the need to develop and quantifv measures of 
effectiveness appropriate to Cc? svstems. According to the MCES principal 
investigator, Dr. Ricki Sweet, MCES is evolving as any scientific development in the 
following steps [Ref. 1: p. 31): 

{. Public discussion and mandate for clarification; 


ae ee up the nature of the problem, the tools, definitions, and potential 
irections; 


First order development of the identified components; 


i = Aad 


Specification of the interrelationships of the components; 


Cay 


Testing of the theory with real problems, 1.e., extra-laboratory experiments; and 
6. Refining the structure in accordance with the test results. 

The MCES methodology is evolving in order to resolve kev C2 issues. 
Throughout this evolution, ce tools and models have been identified, developed, and 
integrated into the MCES. Having completed the first cycle of the referenced six steps 
of scientific development, MCES is now in the process of a continuing iteration of the 
last two steps of test and refinement. The Identification Friend, Foe or Neutral 
(IFFN) Joint Testbed application is an example of such a MCES test and refinement 
iteration. This scientific development will lead to a further refined, bounded and 


generic methodology that may fulfill many C? architecture evaluation requirements. 


D. IFFN BACKGROUND 

The IFFN testbed is currently addressing the air identification problem, which is 
a subset of the overall air defense C* problem. The testbed is representative of the 
NATO air defense C* system which must operate in an environment of friendly, enemy 


and neutral aircraft to perform its air defense mission. Within the air defense Cc? 
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architecture, geographically separated radars and special intelligence sources develop 
detection, track, and identification information on air objects. Computers are then 
used to store and correlate this information. Digital communications are then used to 
share the track information between command facilities. The command organizations 
develop perceptions of the battle situation and make decisions to achieve mission goals 
based on these perceptions. The command organization then implements the decisions 
by directing and controlling air defense forces to take some action against the enemy 
forces. The test concept uses a computer simulation of manned and simulated 
command centers and weapon systems employing real world operating procedures 
against varied threat scenarios. (Ref. 3: pp. 3-5] 

The IFFN C? system bounds were defined by geographic areas of responsibility 
within the NATO Fourth Allied Tactical Air Force (4ATAF) sector. The specific 
command centers that perform the C? functions are: Sector Operations Centers 
(SOC), Control and Reporting Centers (CRC), Brigade Fire Direction Centers (Bde 
FDC), and Battalion Fire Direction Centers (Bn FDC). [Information sources 
considered to be within the C? system are: NATO Airborne Early Warning systems 
(NAEW), Special Information (intelligence) Systems (SIS), and other information 
sources (1.e., flight plans). The air defense ce system architecture included the weapon 
systems when they performed C* functions. The air defense Weapon systems 
considered by the IFFN testbed are the F-15, all weather fighter, and the HAWK and 
PATRIOT surface-to-air missiles (SAMS). 


FE. MCES APPLICATION TO THE IFFN TESTBED 

A MORS workshop team specifically researched the IFFN problem during a 
conference to assist the IFFN Testbed in finding a solution to the IFFN problem. The 
initial C? problem statement formulated by the January 1985 MMORS Workshop from 
the original IF FN Testbed issues was: 


How effective is the Central Europe air defense C2 system in providing decision 
makers the means to assess the sitt ayer and employ air defense assets in order 
to meet overall mission objectives? [Ref. 4: p. 1] 


During the 1985 MORS C* Evaluation Workshop, the IFFN Test Director, Colonel 


Dave Archino issued the following challenge to the working group: 


Develop a tool... specific to air defense that allows IFFN to evaluate the flow 
of C2 information throughout the C2 structure and determine if it is useful or not 
in Winning the war... meeting the mission objectives ... and operational issues 
IFFN plans to address. [Ref. 4: p. 1] 


It was determined that MCES could be tailored to help solve the [PFN testbed 
requirements. Major Patrick Gandee, while a Naval Postgraduate School student, was 


the principal investigator of the \{CES application to the IFFN Testbed. 


F. THESIS ORGANIZATION 

This thesis will summarize how the MCES was specifically applied to the air 
defense problem and how that application has been used by the IFFN testbed to 
address their operational issues. A comparison of the IFFN Testbed and MCES 
approaches will be presented. Since {CES was concurrently evaluated and refined in 
the IFFN application process, this thesis will continue the evolution process by 
formulating recommendations for further MICES refinement. 

The MCES methodology will be outlined in Chapter II]. The IFFN Testbed and 
its evaluation approach will be described in Chapter I{[I. Chapter IV will describe the 
application of MCES to the IFFN Testbed. A discussion of the differences between 
the MCES and IFFN Testbed approach is presented in Chapter V. Conclusions and 
recommendations concerning both the IFFN Testbed and the MCES methodology are 


presented in Chapter VI. 


IH. MCES METHODOLOGY DESCRIPTION 


A. INTRODUCTION 

The following description of MCES is taken in part from Dr. Sweet's report on 
MCES (Ref. I: pp. 10-23] and from her notes and briefings. The figures presented in 
this chapter are revised and updated versions of the ones she used in her publications 
and briefings. A more detailed description and analysis of MCES will be presented in 


Chapter I[V when the MCES application to the IFFN testbed is used as an example. 


B. MCES MODULES 
NICES is divided into seven modules which are detailed below by module. 
1. Module 1: Problem Formulation 

In module 1, the decision-maker’s analysis objectives and needs are described 
for a specific ce problem. First, the decisionmaker’s needs are characterized. The 
analysts consider the decisions being formulated, assumptions about the problem, the 
level of analysis required, and the mission supported. Both the appropriate scenarios 
and assumptions underlying the evaluation are made explicit and the required level of 
analysis 1s determined. This problem statement is then used in the second module to 
bound the C¢ system of interest. The implementation of this module results in a more 
precise statement of the problem and analysis objectives. Figure 2.1 lists the major 
actions required in module 1. [Ref. 1: pp. 10-11] 

2. Module 2: C* System Bounding 

Module 2 identifies the relevant system elements that will bound the svstem. 
When bounding the or, system, a three component definition of a C2 system is used 
based on JCS Publication | |Ref. 5: p. 77]. These definitions state that a Cc? system 
consists of physical entities, structures, and or, processes. Physical entities are 
equipment, software, people and their associated facilities. Structure includes 
Organization, procedures, protocols, concepts of operation and information flow 
patterns. The term "C? process” refers to what the system is doing or the functions 
that the process performs. Bounding the system requires bounding of the physical 
entities and structure. The C* processes are developed in Module 3. 

There are two issues that are raised in this module. The first issue is the 


mapping of Cc? system phvsical components and personnel to the systems boundaries. 
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Figure 2.1 Module 1, Problem Formulation. 


The second issue concerns determining the required levels of analysis. One method of 
graphically illustrating this process of bounding the environment is through the use of 
Ir. Sweet's “onion” as a representative of the environment. The insert of Figure 2.2 
displavs this representation. Starting in the muddle of the onion are the subsystems of 
the C- system. Going out from the middle, the (on subsystems constitute the C2 
system. The Cc system is itself a component of the overall force which in turn is a 
part of the environment. The area outside of the “Onion Skin” ts the rest of the world. 
Successive “peeling” of the “onion skin” will revel the ce subsystems to be evaluated. 
Module 2 results in the bounding of the problem and the identification and 
categorization of the system elements of physical entities and organizational structure. 
figure 2.2 depicts the activities for Module 2. [Ref. 1: pp. 12-13] 
3. Module 3: C* Process Definition 

Module 3 defines the processes needed to fulfill the C* mission. The 
particular command and control process is described by analyzing the generic ce 
processes of the system. The proposed MCES solution to understanding the ce 
processes of a particular ce system is to use an information based paradigm simular to 
the J. Lawson C” Process Model [Ref. 6: pp. 93-99] in the broader framework of 


MCES. The insert of Figure 2.3 displays a modified Lawson's generic Cc process loop 
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Figure 2.2 Module 2, ce System Bounding. 
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model. One example of Lawson’s generic ce loop model components is the ASSESS 
block. The assessments for human decisions are made by battle commanders 
performing the ASSESS function. The commanders’ assessments are made by 
perceiving and assigning meaning to the overall situation. Commanders use 
information from their own sensors, feedback from their forces, and an interface with a 
separate intelligence process to develop these perceptions about the enemy and friendlv 
capabilities. The other functions of Lawson's generic Cc? process model are performed 
in most C? processes. [Ref. 1: p. 14] 

It is necessary to provide a translation of the vocabulary of the Cc? problem 
into the terminology of the generic c? process model to effectively use MCES. This 
translation of terms helps the analyst keep from overlooking critical processes and 
provides standard vocabulary and definitions. In the past, MCES has been able to 
apply the generic or, process model successfully with only minor modifications. There 
are different C- process levels and interactions among the ce process function 
components and these process relationships can become very complex. To illustrate 
different levels of processes, an example of a commander performing a decision 
function can be used. The commander passes his decision to several subordinates who 
in turn work out detailed instructions to implement that decision. These subordinates 
communicate the instructions to their forces which then act in the environment. In 
command and control terms, the commander and the subordinates were performing 
separate decision functions within a Cc? process. The subordinates’ decision function 1s 
related to the commander's decision function by the commander's decision (output) 
and the subordinates’ receipt (input). In turn, the detailed instructions from the 
subordinates to the force couple the subordinate’s function to the force function. This 
functional input/output relationship forms a “structure” between separate Cc process 
functions which are required to perform the mission. The structure determines the 
information flow. [Ref. 1: p. 15] 

In a distributed C? system, processes may be related to other processes. The 
processes that have been valuable to MCES C* evaluations for describing distributed 
information flow are [Ref. 1: pp. 70-73]: 


|. Intelligence (INTELL) Process.. Assigns meaning to observed activities and 
situations and forecasts changes in the current situation. 


2. Crosstell Pee Process... A subset of the communications process, which 
provides for sharing of information throughout the C2 system to support 
decisions and their implementation. 


3. Execution Level C2 Process. Directly controls weapon systems. 
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‘\ complete picture of the ony svstemis architecture includes the intelligence process and 
how it interfaces with the C* process. NTELL is the sharing of informationsanaues 
needed to describe the coordination between distributed C? systems. At a single 
command node, XTELL 1s simply a process function but on a systems level it is a 
networking process. At a single node, the Cc? process directly controls weapon 
systems. At a higher level, the Cc? process coordinates the directing of the weapon 
systems. These processes are dynamic descriptions of what the ce system is doing. 
NITELL and INTELL processes interfaced with the C? process in a distributed system 
are ultimately linked to the weapon systems which perform the mission. 

Dr. Sweet emphasizes that this module forces the analysts attention on 
(Rete po. 14): 


|. the environmental cause or initiator (i.e., the enemy force) of the C2 process 
that results from a change in the desired state; 


tN 


the internal C2 process functions that characterize what the system is doing 
such as sense, assess, generate, select, plan, and direct; and 


Ud 


the input and output from the internal C2 process that couples with the force 
process 


Figure 2.3 represents the major actions required in module 3. As a result of the 
implementation of this module, the functions of the Cc? process for a given problem are 
identified and mapped to the generic ce process loop. 

4. Module 4: Integration of System Elements and Functions 

Module 4 relates the information flow to a C¢ system by means of its Cc 
process functions. Functions are subsets of the or, process and represent what the C2 
system actually does or accomplishes. The relationship of the Cc? phvsical entities to 
the process functions and organizational structure is also formulated in this module. 
This integration is accomplished by making explicit the relationships between these 
components. Figure 2.4 outlines the actions required in Module 4. 

The first step is to map the physical entities of man and machine which 
perform the functions and produce output to an organizational structure. All ce 
functions can be potentially performed in a single node or be distributed between 
different nodes so this mapping results in an organizational structure which graphically 
depicts a single node or a distribution of command and weapon nodes depending on 
the svstem’s unique configuration. 

Next, the flow of information is charted by techniques such as Data Flow 
diagrams (DFDs) [Ref. 7: pp.99-115] or Petri Nets [Ref. 8] which may be used to 
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Figure 2.4 Module 4, Integration of System Elements and Functions. 


descnbe information flow through the Cc process model. DEDs and Petri Nets will be 
discussed in greater detail in Chapters V and VI. DFDs are an example of a technique 
that has been productive in this module in determining relationships between processes 
and information flow. Data Flow Diagrams (DFDs) are first constructed to show 
information flow through the Cc? process model. The inputs and outputs of each 
function are determined and related to the other functions in the process. In the next 
step, a transforin analysis 1s performed to uncover the transform center (process center) 
and to determine the subordinate and superordinate relationships between the 
transform center and the individual C? functions. Information flows into the 
transform center and control information flows out of the transform center. These 
input. output relationships describe the internal information flow between separate 
process functions. The data flowing into the transforin center or process center 1s 
information flow while the data flow out of the transform center is control information 
flow in the form of action requests or conimands. Thus a hierarchical “structure” has 


been defined in terms of the mission essential information flow between functions 


within the C- process. From this information, a Cc? system architecture will eventally 
be formulated. These procedures will result in the description of how the elements and 
plavers function together, which 1s basically relating information flow to organizational 
structure. [Ref. l: pp. 16-17] 

5. Module 5: Specification of Measures 

\fodule 5 specifies the measures necessarv to evaluate the Ce problem. These 
measures are then classified as to their level of measurement, i.e., dimensional 
parameters, measures of performance (MOPs), measures of effectiveness (\{OEs), or 
measures of force effectiveness (N[OFEs). MOFEs are used to describe the actions 
between the force and the environment. When the C? system is combined with the 
force, the environment will be effected and MMOFEs measure this force effect. Within 
the force boundary, MOEs are used to measure how the combat force is effected by the 
Cc? Operation. \I{OPs are applied at the ce system boundary and measure how well the 
Cc? svstem performs its functions. For the subsystem within the boundary of the Cc? 
svstem, dimensional parameters are used to measure the limits of the subsvstems. The 
resulting measures may be used to determine differences in a ce system When utilizing 
alternative configurations of the phvsical entities, structure, or processes. Figure 2.5 
graphically depicts the “onion” with its corresponding measures and the actions 
required in Module 5. 

This MCES implementation results in the specification of a set of measures 
focused primarily on the process functions. The process functions identified may be 
used to derive a complete set of relevant measures which can then be subjected to 
further scrutiny. A set of measures can be compared to a set of desired measures 
characteristics as shown in Table 1 [Ref. 1: p. 20] to insure that the measures are 
useable. 

6. Module 6: Data Generation 

In Module 5, one of several types of data generators such as exercises, 
experiments, simulations, subjective judgements, or relevant experiences 1s selected to 
generate the necessarv values for the measures formulated. These values may be either 
measured directly or indirectly. The analvsts consider the reproduciblity of results, 
precision and accuracy, timing of collection, environmental controls, and experimental 
design during this module. A timeline is formulated to set the completion dates for the 
data generation phases. Using the designated data generator, the resulting values for 
these measures constitute the output of this module. Figure 2.6 outlines the data 


generation module. [Ref. 1: p. 21] 
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Figure 2.5 Module 5, Specification of Measures. 


TABLE’ 
CRITERIA FOR EVALUATION MEASURES 


CHARAGTERI SHG: DEFINITION 


Mission oriented Relates to force/system mission. 


Pirserimina corny Identifies real difference between 
alternatives. 

Measurable Can be computed or estimated. 

Quantitative Can be assigned numbers or ranked. 

Realistic Relates realisticly to the ce 
system and associated uncertainties. 

Objective Can be defined or derived, 
independent of subjective opinion. 

Appropriate Relates to acceptable standards and 
analysis objectives. 

Sensitive Reflects changes in system variables. 

Inclusive Reflects those standards required 


by the analysis objectives. 


Independent Is mutually exclusive with respect 
to other measures. 


Simple Is easily understood by the user. 


7. Module 7: Aggregation of Measures 

Module 7 1s the final module and addresses the issue of the aggregation and 
interpretation of the observed values of the measures. Figure 2.7 depicts the 
aggregation and interpretation process. From data generation, values for the identified 
measures will be obtained and analvzed. One of the analysts’ concerns will be to relate 
command and control systems to some measure of force effectiveness which is 
sometimes termed the force multiplier effect. For MOFEs, the intent of aggregation is 
to relate the C¢ system with combat svstems to indicate combat outcomes. After 
aggregation, the issues of measure causality, sufficiency, and independence are to be 
considered. Scenario dependence must also be addressed. Because combat is very 
complex, many measures will not show significant differences. Analysis must be 


conducted to determine the important factors in the particular scenarios. At this 
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Figure 2.6 Nflodule 6, Data Generation. 


crucial point, the analysts must decide if their questions can be answered bv their 
analvsis. Credibilitv. and reliability are major concerns to the decisionmakers. 
peer 1: pp. 21,22 


C. COURSES OF ACTION 

The results of the \ICES iteration are provided to the decisionmaker. Figure 2.8 
represents the actions and results of each iteration of the MCES modules. At least two 
courses of action are then available to the decistonmaker based on the results. The 
decisionmaker may directly implement the results of the NICES evaluation or he may 
identify the need for further study or require another iteration of the MCES analysis. 
The decisionmaker may interact with the \{CES analysis eflort to further guide the 
analvsts by identifying errors in assumptions, clarifying the bounding, etc. The analvsis 
could be modified by infusing new directions or objectives based upon the results of 
previous MICES modules completed. [or example, the bounding of the Cc system) may 
generate the observation that significant interfaces are outside the originally conceived 


scope of the study and require a return to the problem formulation module. 
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Figure 2.7. Module 7, Aggregation of Measures and Interpretation. 


D.- USES 

\ICES can assist in the areas of C? management and analysis. MCES assists in 
the systematic specification of the problem by focusing on tdentifted essential 
characteristics of the C* system. It permuts a senior analvst to conduct a on 
evaluation effectively. MICES assists the analyst in forming a concise conclusion and 
provides the manager with supporting data for decisionmaking. The IIT PN Testbed 1s 
a good example of a C* evaluation of air defense and will be illustrated in the following 


chapters. 
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It. IFFN TESTBED DESCRIPTION 


A. THE IDENTIFICATION PROBLEM 

The Identification Friend, Foe or Neutral (IFFN) Testbed, compmsed Ofvamiees 
Army and Air Force Joint Test Force at Kirtland Air Force Base, is investigating ways 
to enhance the identification of friendly, neutral, and enemy aircraft. A realistic 
scenario used by the IFFN Joint Test Force forecasts that in future air battles our 
tactical air defenses will be faced with sophisticated enemy fighters capable of engaging 
our forces with bevond visual range (BVR) weapons The large number of enemv 
aircraft with their use of low level tactics, high speeds, and Electronic Countermeasures 
(ECM) challenges our air defense systems. The air defense system must be able to 
identifv and characterize the enemy attack and then direct sufficient force in time to 
neutralize it. [Ref. 9: p. 47] 

Effective performance of the active air defense mission requires a capability to 
correctly identifv aircraft in a timely manner in order to facilitate the air defense 
Weapon system's ability to emplov its weapons. This air defense process is done 
through a complex arrangement of personnel, equipment, communications facilities 
and procedures which form a ce system. This requirement is particularly important in 
the European theater where large numbers of friendly, neutral, and enemy aircraft will 
be part of the tactical air environment. In this environment, surface-to-air and air-to- 
air Weapon systems must Operate in conditions beyond ranges where positive visual 
identification can not be performed. The problem is aggravated when modern 
electronic warfare (EW) threats, particularly those of the Warsaw Pact are considered. 
Numerous studies have revealed that current electronic identification capabilities have 
numerous problems including being too slow, poor at positively identifying enemies 
and friends, insufficient range capability, and being subject to interference and 
electronic countermeasures (ECM). The inability of air defense weapon systems 
Operators to accurately and rapidly discriminate friend, foe, or neutral aircraft results in 
the ineffective use of these weapon systems. These problems have stimulated activity 
within the US military and NATO to develop an effective NATO identification system. 
The IFFN Testbed is a partial attempt by the United States at solving this air defense 


Problems: (Reiss pw) 


The function of acquiring, correlating, fusing, and disseminating direct and 
indirect IFFN information is an wunportant part of air defense C*. This IFFN 
information serves as a basis for threat evaluation and engagement control. It is also 
the function of air defense command and control to provide sufficient identity 
information, bearing, and range to the allocated weapon svstem so that the weapon 
system is able to acquire and engage a target with a high degree of confidence. 
Acquisition and engagement information processed in a tmelvy manner will permit 
effective weapons employment. An IFFN system should also provide the necessary 


information to allow passage of friendly and neutral targets. 


B. THE AIR DEFENSE PROCESS 

Identification is an integral step in the air defense C2 process and begins with 
tasking and disposition of assets for air surveillance. The process continues through 
detection of an intruding target and ends with a decision to engage the target with an 
air defense weapons system. This process is characterized by complex relationships 
between air surveillance, C7, and Weapons systems. The targets will be identified as 
friendly, neutral, hostile, or unknown. The hostile targets will be given different 
priorities for engagement or mav be engaged by other air defense assets. Air defense 
C* must have sufficient identity information to evaluate the intent of hostile targets 
and assess the threat level posed by those individual targets. Air defense C* must then 
prioritize those targets for engagement, allocate the targets for engagement by specific 
Weapons systems, and aid a weapon system in acquiring that target without 
endangering other targets. This must be done in a constantly changing air 
environment where identity determination is a dynamic process involving people, 
hardware, and software. 

The classical sequence of air defense is detect, identify, engage, and destroy. This 
classical sequence used by the IFFN Testbed is a simplification built from a number of 
decisions and functions performed separately or mutually by C* elements and air 
defense weapon svstems. Most of these functions and decisions are dependent upon 
identification. Complicating this simple sequence is the fact that identification is both 
a process and a decision. As a process, identification is a constant gathering and 
correlating of information about a potential target from all sources of direct and 
indirect identification information. This process is continuous up to the final 


disposition of a target by the air defense system. As part of the identification process, 
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the C? svstem must correlate information from all sources and resolve anv conflicts. As 
a decision, an identification must be made before further action can occur, either 
actively or by default. This decision is influenced by all the sensors available to the air 
defense svstem, by the background intelligence on the air situation, and by the 
operational procedures such as rules of engagement and weapons control status. The 
decision-maker must either make or delegate the identification decision prior to the 
engagement decision. [Ref. 3: p. 8] 

When performing the classical air defense sequence within an integrated air 
defense system, the identification decision is also part of a larger process which is 
performed by both C” elements and air defense Weapons systems. The process 
becomes more complex when additional sources of direct and indirect information are 
available and higher levels of command and control participate. Individual weapon 
systems are simultaneously performing detection, tracking, and identification as part of 
their target acquisition function. They can be aided in performing this function by the 
command and control system as it exercises its function of engagement control. When 
Operating autonomously, Weapon svstems are limited to their organic detection, 
tracking, and identification capabilitv. The detection, tracking, and identification of 
the entire svstem can be better used for target allocation and acquisition when the 
command and control svstem provides engagement control in a centralized mode of 
operation. Identification 1s a major factor in the performance of Weapon systemis to 


defeat the enemi: 


C.  IFFN JOINT TESTBED CONCEPT 

The Department of Defense’s proposed partial solution to the NATO air defense 
problem was the development of the IFFN Joint Testbed to gather analvtic data on 
the problem so that solutions could be formulated. The testbed will assess baseline US 
capabilities within the NATO air defense ce system to perform the IFFN function, 
identify deficiencies in the performance of that function, and propose potential near- 
term procedural and equipment modifications for further testing. One issue that will be 
addressed by the IFFN Testbed is the indirect information process and how its use 
mav improve the performance of air defense systems to aid C* and Weapon systems 
nodes. A testbed approach was taken so that a number of different on, Strategies could 
be tested without having to actually use the real equipment and weapons. Ihe testbed 
was envisioned to simulate as close as possible the real threat and the U.S. equipment 


and procedures used in NATO. [Ref. 3: pp. 1,2] 


The level of complexity of the air defense ce system is enormous, thus, the LFFN 
Test Force expended a large amount of effort in understanding general on svstems 
before modeling the NATO air defense ce system. A simulation testbed was ultimately 
chosen to evaluate the alternative air defense Cc? swsteiis. ihe (PrN fest Force is 
attempting to determine air defense identification measures of performance (MOPs) 
and measures of effectiveness (MOEs) that will lead to the evaluation of the utilitv of 
the different configurations of the air defense C? architecture. 
1. Baseline Architecture 
The IFFN baseline architecture was formulated as a combination of hardware, 
software, procedures, and doctrine that is planned to exist in the late 1980's. The 
criteria will also be subjected to the projected Warsaw Pact 1987-1990 threat and is 
consistent with Defense Intelligence Agency estimates of enemv capabilities and orders 
of battle for that period. The timeframe chosen was a compromise between possible 
near-term benefits and results which will have long range applicabilitv. Certain 
modifications that will be fielded in 1987 or beyond will be candidates for follow-on 
Pests using tae [FPN testbed. [Ref. 3: p. 3] 
The testbed geographical area of interest is the NATO environment in which 
US Army and Air Force units operate jointly and under the control of associated 
elements of the NATO Air Defense Ground Environment (NADGE) System. The 
IFFN testbed will focus on the battle management area of a representative NATO 
Control and Reporting Centers (CRC) located in the Forth Allied Tactical Air Force 
(JATAF) area. Representation of key associated NATO command and control nodes 
and information sources is required in the IFFN testbed. Figure 3.1 depicts the key 
components of the IFFN Testbed. [Ref. 3: pp. 3-6] 
a. Command and Control Units 
Command and control units are those representative units which direct or 
control the beyond visual range (BVR) weapon systems and execute the active air 
defense mission. The specific command centers that perform these C* functions are: 
Sector Operations Center (SOC), Control and Reporting Centers (CRC), Brigade Fire 
Direction Centers (Bde FDC), and Battalion Fire Direction Centers (Bn FDC). 
b. Information Sources 
Sources which provide information for identification, target allocation, and 
target acquisition in the air environment to the C? units and weapon systems are 


categorized as information sources. These are: NATO Airborne Early Warning 
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svstems (NAEW), Special Information (intelligence) Systems (SIS). and other 
informatuon sources (1.e., flight plans). 
c. Weapon Systems 
Weapon svstems are those representative BVR weapon svstems which are 
operated bv US forces. For the IFFN Testbed, the F-15, FIAWh, and P17 Ri@iiiaae 
been selected as the representative BVR weapon systems. Due to resource constraints 
on the [FFN Testbed, onlv these three weapon systems will be used. 
2. Major Operational Issues 
The testbed will generate and record the data necessarv for analysis and 
recommendations on IFFN issues There are three major operational issues considered 
by the [FFN Testbed which are listed belenz 
a. Issue I 
“What is the contribution of indirect identification information to the 
ability of US air defense command and control systems operating in NATO to 
correctly identify airborne targets, to use identification in performing target allocation, 


and to aid subordinate air defense weapon systems in performung target acquisition?” 


Us 
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[Ref. 10: p. 2]. Indirect ID information is that ID information that is not direct 
electronic IFF returns. Indirect information usually refers to all other ID information 
that arrives as a facility such as flight plans, area of operations, intelligence reports, 
etc. However, direct ID information once passed to another facility also becomes 
indirect [D information. Satisfying the first major operational issue will provide a 
baseline assessment of the expected identification performance of a representative 
NATO air defense system operating in the 4ATAF area. Studying the first issue will 
also provide a fuller understanding of the relationship between the identification 
performance of the ce system and the performance of the overall active air defense 
nussion. 

b. Issue 2 

“What are the deficiencies in air defense system use (collection, formation, 

dissemination, and use) of indirect identification information for which solutions are 
not currently planned?” [Ref. 10: p. 2]. The second major operational issue will 
identify weaknesses in the identification process and allow for a qualitative comparison 
of those weaknesses identified during testing. Potential corrective actions for these 
identified deficiencies can then be developed. These recommended corrective actions 
could take the form of changes in doctrine, procedures, system software, 
communications connectivity, or addition of new data sources, or various combinations 
of these solutions. 

c. Issue 3 

“What near-term procedural and equipment modifications should be 
recommended to overcome deficiencies?” [Ref. 10: p. 2]. This issue will address 
productive near-term solutions to the IFFN problem. 
3. Hybrid Approach 

Two major options were considered during feasibility studies when developing 
the test concept. Field exercises and computer-based simulation were both evaluated 
and compared. A hybrid approach was ultimately selected which permits the use of the 
best of both options. The concept is centered around live operators using actual 
tactical hardware and accepted simulations of hardware and software called Live 
Participating Units (LPUs). Real-time computer models stimulate the LPUs as well as 
represent the background workload for these units. This man-in-the-loop simulation 
will be carried out through all the tests. Since the IFFN Joint Test Force developed a 


hybrid simulation testbed composed of simulated participating units, there was no 
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requirement for live operation of aircraft, weapons, radars, nor field deplovment of 
weapons and conimand and control systems. To implement this test concept a 
distributed testbed was established at a central facility at Kirtland Air Force Base, New 
Vlexico. The testbed at Kirtland generates and distributes the tactical scenario, 
controls test execution, and monitors the response of geographically distributed Patriot 
and Hawk LPULs at the Army Air Defense Center at Fort Bliss, Texas. 

A realistic scenario environment was necessary to ensure realistic and accurate 
results. The combination of a few high fidelity LPUs responding to simulated and 
manned units was determined to be an excellent method to simulate the 
interdependency and interaction of air defense units. The IFFN Test Force conducted 
a substantial testbed certification effort with joint service participation to validate and 
calibrate testbed performance against joint and combined field exercises for the first 
test series. Further credibility and validity tests will be made after each test series. 

4. Models 

The IFFN models that were developed can be categorized as interactive or 
noninteractive. The interactive models react dynamically to perceived changes in the 
air battle situation. Thev mav receive inputs such as data link messages from the other 
models or LPUs and may initiate messages in response to this input or their own. The 
output of these models is dependent on the specific dvnamics of the air battle. The 
applications of the interactive models for the IFFN testbed are: sensor models, missile 
models, and dvnamicallv controlled aircraft models. Examples include radar, electronic 
[IF F, Patriot SAMs, and fighter and attack aircraft platforms. [Ref. 10: p. 10] 

The IFFN noninteractive models do not react to the air battle dynamics. 
They are less complex models and simply generate selected messages and actions at 
preprogrammed times according to a script prepared prior to the test. The models are 
considered suitable for simulating those facilities that do not dvynanucally interact with 
the identification process, but do provide orders, procedures, and other information on 
a one-way basis such as certain higher echelon planning facilities. Noninteractive 
models will also be used to simulate aircraft following programmed flight profiles which 
are not automatically reactive to the air battle environment. The Sector Operations 
Center (SOC) is an example of a noninteractive model used in the IFFN Testbed 
simulation. Examples of weapon platfornis are transport, patrol, and tanker aircraft. 
[Ref. 10: p. 14] 


D. TESTBED PHASES 
In order to minimize technical and program risks, a phased testbed acquisition 
was adopted. The test approach 1s based on seven test series. The series will consist 
of weapons systems, command and control systems, and associated data links. One of 
eight planned phased simulations has been completed. The following is a description 
and list of the svstems involved. [Ref. 10: pp. 3.4] 
1. Test Series 1 
Series 1 tests the identification performance of a representative US Army 
Surface-to-Air Missile (SAM) Svstem which 1s the PATRIOT fire unit. The svstems 
tested are: 
Pera wRiOt Fire Unit (FL), and 
2. PATRIOT Air Defense Information Language (PADIL). 
2. Test Series 2 
Series 2 adds the PATRIOT’s first echelon of command and control, the 
Battalion Fire Direction Center. The systems tested are: 
Pema tT RIOT PU, 
2. PATRIOT Battalion Fire Direction Center (Bn FDC), and 
Bee) 
3. Test Series 3 
Series 3 adds the next level of C2, the Brigade Fire Direction Center. The 
systems tested are: 
foe eRe tT FL, 
Pee tRl@ 1 Bo. FDC, 
3. PATRIOT Brigade Fire Direction Center (Bde FDC), 
4. PADIL, and the 
Army Tactical Data Link-1 (ATDIL 1). 
4. Test Series 4 
Series 4 tests only the USAF’s fighter-intercepters, the F-15 . The system 


in 


tested is the F-15 “Eagle” Intercepter. 
5. Test Series 5 
Series 5 adds associated USAF C* nodes and information sources to the F-15 
system. The systems tested are: 
1. F-15, 
2. USAF Control and Reporting Post/ Message Processing Center (CRP‘' MPC), 
3. NATO Airborne Early Warning System (NE-3A), 
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6. 


Special Information Svstem (SIS), 
Tactical Digital Information Link - A (TADIL-A), and 
TADIL-B. 


6. Test Series 6 


Series 6 will integrate the Army systems from Series 1-3 with the USAF 


systems from Series 4 and 5. This will now be a joint operations test. The systems 


tested are: 

lo SPATRI© Tae 
2 PATRIOD Bn ee 
3. PATRIOT Bde FDC, 
Ae) Flo, 
5. NE-3A, 
Of. “CRE. 
72 Sls. 
8. TADIL-A, 
oo NOE. 

10. PADI. 

11. ATDL-1, and 

12. NATO Link-1. 


7. Test Series 7 


Series 7 will add a CRC to form the total system to be tested. The svstems to 


be tested are: 
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PATRIOP Fk, 
PATRIOT Bn FDC, 
PATRIOT Bde FDC, 
F-15, 

NE-3A, 

CRE 

Sls; 

NATO Control and Reporting Center (CRC), 
TADIL-A, 
TADIL-B, 

PADILE, 

ATDL-I, and 

NATO Link-l. 
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FEF. IFFN TEST CELL MATRIX 

The simulation will be conducted using a controlled variable approach. Different 
simulation test cells are used in which some variables are held constant while others are 
left to fluctuate and eventually lead to the determination of the variables’ impact on 
the C? system. The basic test structure considers both a fully integrated air defense 
system and several subsets of the svstem. The different configurations allow different 
variables to be isolated to establish their contribution to the particular air defense C? 
system. Each trial will be run with a constant measurement volume of a prescribed 
radius with only predetermined variables changed. The measurement volume 1s the 
area covered by the air defense unit or weapon system. The test cells of the matrix are 
used to generate comparative data under various environmental conditions. Figure 3.2 
represents the basic test matrix of ten test cells used by the IFFN testbed. 

Data items are collected from collection messages that are generated by data 
events during the simulation. Data events are events that take place during the 
Simulation such as information arrival and actions performed by the air defense Cc? 
system nodes. For Test Series 2, there are fourteen data events that are collected from 
each test cell simulation to be used in the calculation of the MOEs and MOPs with 
other data events used for deficiency analysis. All MOEs and MOPs are divided into 
two groups of probability measures and distribution measures. These measures are not 
necessarily measures of the variables themselves but are intended to be measures of the 


results of the variables impact on the ce system. [Ref. 11: p. 20] 


FL. MEASURES 

General categories of measures were sought to derive values that would 
eventually lead to discrimination among the different C2 systems. The simulation can 
not completely characterize the performance of the fully deploved air defense svstems, 
so absolute conclusions about the performance of the air defense systems are nearly 
impossible. With this shortcoming in mind, measures of the relative change in the 
performance effect of the variable under varying conditions will be used when only 
large and significant differences are noted. This means that a low confidence level will 
be used when analyzing the data. Figure 3.3 depicts the general approach taken by the 
IFFN Testbed in resolving its [FFN issues. 


Be 


HIGH LOW 
ACTIVITY ACTIVITY 
LEVEL LEVEL 


Ps $s 


i 


VARIANT 
ACP's 


CONTROL 


DECENTRAL 
CONTROL 


ee 





¢ NOM (Nominal) 

- ACP’s (Airspace Control Procedures) 

¢ Q&A IFF (Question and Answer Electronic IFF) 
- FU (Fire Unit) 

- HD (indirect Identlficatlon) 

« AUTO ( Autonomous) 


Figure 3.2 IPFN Basie lest Sian 


38 


IFFN APPROACH 
(Without MCES) 


IFFN ISSUES 


Vy 


TEST OBJECTIVES 


Vy 


TEST DESIGN 


Test Matrix of Competing Configurations 
Simulation Testbed 


\) 


SPECIFICATION OF MEASURES 
Air Defense Functions —» MOE 


Subfunctions —~», MOP 


Vy 


DATA GENERATION 
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1. Test Series 2 Objectives 


There were originally six objectives formulated from the three issues for Test 


Series 2 that eventually led to the formulation of the IFFN Testbed measures. These 


objectives are listed below without the more detailed sub-objectives for each objective. 


l. 
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PECUN 1 - Assess and contrast the performance of the PATRIOT battalion 
under centralized and decentralized control. 


Objective 2 - Assess the impact of changing and removing Airspace Control 
Procedures (ACPs) on the operational performance of the centralized and 
decentralized battalion and autonomous fire unit. 


DEMING 3 - Determine the value_and impact of perfect ID and communication 
system performance on PATRIOT battalion performance. 


Recut 4 - Evaluate the impact of various changes in direct and indirect ID 
performance and interaction. 


Objective 5 - Assess the influence of the fire unit ID function on the ablitv of 
the PATRIOT battalion to perform its functions. 


Objective 6 - Identifv and subjectivelv evaluate any PATRIOT operational 
egiiciencies noted during testing, (Ref. Ik: pp. 2.1,2.2} 


oe 


2. Measures of Effectiveness (NIOE) 

There are three primary MOE areas formulated for all the test series. The 
IFFN Testbed originally identified three functions that were performed in the air 
defense process: identification, target allocation, and target acquisition. [Ref. 3: pp. 
17,18] 

a. MOEs for the [dentification Function 

These MOEs will describe how well the weapons and Cc? svstems are able 
to identifv or recognize airborne objects and assign them to appropriate identification 
categories. 

b. MOEs for the Target Allocation Function 

These \fOEs will relate identification information used by Cc? systems to 
allocate air defense Weapons against hostile aircraft and prevention of misallocation of 
weapons against friendly aircraft. 

c. MOEs for the Target Acquisition Function 

These {OEs provide the measures relating indirect identification 
information to the weapons svstems. Target acquisition provided by the command and 
control structure to the weapons systems is a part of these MOEs. 

d. MOE List 

The MOEs for Test Series 2 that measure the needed information for the 
C* evaluation issues are listed in Table 2 [Ref. ll: p. D25]. The letter “P” tdemenias 
probability measures while the letter “R” signifies range distributions and the letter “T” 
represents time distributions. 
3. Nfeasures of Performance (MIOP) 

Measures of Performance (MOP) for the [FFN Testbed were submeasures of 
the air defense functions and therefore subsets of the MOEs. An example is the 
probability that a passed ID is correct which is a submeasure of MOE 3, probability of 
identification of an aircraft. The MOPs for Test Series 2 that were determined to 
measure the needed quantities or qualities to resolve the six stated objectives are listed 
in Table 3 [Ref. 11: pp. D23,D24] with corresponding MOE number references. 

4. Measures of Force Effectiveness (NIOFE) 

Measures of Force Effectiveness are global measures that determine how 

effectively the mission is accomplished. Target hits and damage assessments were not 


modeled in this testbed so MIOFEs could not be used. 
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MOE# MOE 
ee e( D/O 
Pee L/D) 
a. MPP 1/5) 
4. P(A/T) 
5S. P(E/A) 
P(E/V) 
P(E/F) 
8. P(E/N) 
9. P(E/H) 
10. T(Tot) 
11.  P(EBO) 
12. R(FE) 


TABLE 2 
MOE DEFINITIONS 


Pere Pee rNI TION 


Probability of detecting an aircraft 
given that it has entered a system 
measurement volume. 


Probability of tracking an aircraft, 
given that it has entered a system s 
measurement volume and has been detected. 


Peobabi lity of identification of an 
aircraft, given that it has entered 

a system s measurement volume and has 
been tracked. 


PEcoaowuty Or allocation of an 
aircrate, given that it has entered 

a system s measurement volume and has 
been identified. 


Bao a Od bat ViaO o Sega nene of an 
aircraft, given that it has entered 
a system s measurement volume and 
has been allocated. 


Probability of engaging an aircraft. 


Probability of engaging a friedly 
al Gerat tc. 


Probability of engaging a neutral 
airerart. 


Probability of engaging a hostile 
alee rat tc. 


Distribution of times elapsed between 
detection and engagement. 


PRooabr lity Of ead a hostile 
aircraft before i ires 
a missile or drops ordnance. 


Dieu lowmOresanges From alrcraft to 
FSCL at time of engagement. 


4] 


| | 
TABS | 
MOP DEFINITIONS 


MOE 

ref# MOP MOP DEFINITION 

1) R( D) Distribution of ranges from airerams 
to detection unit at time of detection. 

1) R( FD) Distribution of ranges from alrcrafeees 
FSCL at time of detection: 

2) R(T) Distribution of ranges from alrcrateues 
tracking unit at the time the track was 
established. 

2) R( ET) Distribution of ranges from aircraft to 
FSCL at time of tracking. 

2) an) Distribution of times elapsed between 
detection and tracking. 

S)) P(IX/Y1I) Probabilities of identifying an airerage 


as category Aq friend, neuer aly 

hostile}. given that_1ts true identit 
is (friend, neutral, or hostile) an 
that it has been detected. 


3) RG) Distribution of ranges from aircraft to 
identifying unlG at Erimeg@er é 


3) R( FI) Distribution of ranges from aircraft to 
PSCL ate tine cilwip: 

3 ) (cabal) Distribution of times elapsed between 
tracking and ID. 

3) P( Pass) Probability that a passed ID is correct. 

35) P( Res) Probability that an ID Gomeliceer. 


resolved while the aircraft is stile 
the weapon system s measurement volume. 


3) T( Trans) Distribution of times elapsed between 
receipt and retransmission of ID 
information by a C2 node. 

3) P( Amp ) Probability that an ID includes track 
amplification information. 

4) P(A/YI) Probabilities of allocating an alrecrame 


liven that its true identity is 
friend, neutral, or hostile) and that 
it has been identified. 


4) R( A) Distribution of ranges from aircraft to 
allocated unit at time of allocation. 


4) R( FA) Distribution of ranges from alrcraitees 
FSCL at time of “allocate 


TABLE 3 
MOP DEFINITIONS (CONT'D. ) 


T( IA) Distribution of times elapsed between ID 
and allocation. 


P(E/YA) Probabilities of enga ae es alEGrat t, 
1 


iven that its true identity is Y 
friend, neutral, or hostile) and 
hat it has been allocated. 


Rens Pum rLOnmucm Landes from alrcraft to 
engaging unit at time of engagement. 


T( AE) Distribution of times elapsed between 
allocating and engagement. 





G. ANALYSIS OF DATA 
1. Exploratory Data Screening 
The test data results are first examined and screened for outliers and to verify 
underlying assumptions such as normalitv, independence, constant variances, and zero 
mean. Various methods are used to statistically check the data. Data screening 
involves a number of methods listed below. [Ref: 11: pp. D37-D42] 
a. Box and line plots 
Box and line plots are used for time and range distributions. The box or 
line plot allows pictorial presentation of a set of distribution measures for a set of trials 
or number of test cells. 
b. Frequency distributions ( Histograms) 
Histograms are used also for time and range distributions. These show 
visual evidence of normality as well as extreme outlying values. 
c. Scatterplots 
Scatterplots are used for time versus range plots distributions. Bivariate 
scatterplots provide a visualization of the relationship between two continuous 
variables. 
da. Normal probability plots 
Probability plots are used to determine probability types and fits. Normal 
probability plots provide visual evidence of the difference between a given distribution 


and a Gaussian distribution. 
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2. Data Analysis 
The data analvsis that follows screening 1s listed below. 
a. Paired T Tests 

Means are tested using this test. The assumptions concerning the 
underlving populations are that thev are independent samples and the variances are the 
same. If they are not, then & festssareausede 

b. Analysis of Variance (ANOVA) 

ANOVA allows inferences to be formulated about differences in “treatment 
effects” brought about by the test variables which are controlled from cell to cell. 
Inferences are made by estimating how much of the variability in test data is explained 
bv the effect of the test variables and how much is due to random error. 

c. Hypothesis Testing 

The ANOVA provides a basis for the formal hypothesis test that the trial/ 

cell means or specific subgroup means are all equal. 
d. Contingency Table Analysis 

This analysis is sometimes called row by column (RC) table analysis. This 
test is used when more than two outcomes are possible, a frequent occurrence among 
the test cells. Each 2x2 contingency table analysis will be performed for comparing 
probability measures from cell to cell on as many of the measures as practicable. 

e. Regression 

Regression analysis determines the statistical relationship between one or 
more independent variables and a dependent variable. Curve fitting is accomplished bv 
regression of the dependent variable on the independent variable. 

f. Correlative analysis 

The relationship between the independent and dependent variables or 
causalitv is determined by correlation analysis. This analysis determines what 
proportion of the variation of the dependent variables can be attributed to the 
relationship with the independent variable. 

g. Standard Normal Theory Approximations 

These series of tests are used to determine what type of probability 

distribution exists and how good that data fits that distribution. 
h. Deficiency analysis 
After the data has been checked and analyzed, the causes of the differences 


in the C¢ configurations are proposed and examined. The objective is to find the 
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underlying cause or reason behind the differences. This could be helpful to 
decisionmakers in allocating limited resources to different Cc configurations. 
[Ref. Ll: pp. Dl1-D29] 


H. IFFN PROGRESS SUMMARY 

It is evident that the IFFN Testbed has made progress in its attempt to evaluate 
the NATO air defense C* system. The IFFN air defense problem is definitely complex 
and the IFFN Testbed has understandably committed large amounts of resources to 
the problem. The Testbed is a good concept for an experimental design to test 
competing Cc? svstems since all of the alternative svstems can not be tested using actual 
equipment due to resource constraints or present Cc? configuration limitations. The 
IFFN Test Force has been careful to insure that the testbed is credible. Only Test 
Series | has been completed with Test Series 2 to begin in March 1987. 
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[V. PROPOSED MCES APPLICATION TO THE IFFN TESTBED 


A. INTRODUCTION 

Primary research into the application of MCES as an evaluation tool for the 
IFFN Testbed was undertaken by Major Patrick Gandee, U.S. Air Force, when he was 
a Naval Postgraduate School student. Two Military Operational Research Society 
(\fORS) teams also contributed to the proposed MICES application during two MORS 
conferences. The bulk of Major Gandee’s and the MORS teams’ work with the IFFN 
Testbed is included in Major Gandee’s thesis [Ref. 12] and later refinements by Major 
Gandee as a staff member of the Joint Chiefs of Staff Command and Control Systems 
Directorate [Ref. 9: pp. 49-58]. Dr. Ricki Sweet, who was Major Gandee’s thesis 
advisor, provided the then current MI1CES methodology guidance. This application 
study was also supported by the Naval Postgraduate School and the Office of the Joint 
Chiefs of Staff. Major Gandee received assistance from IFFN Testbed personnel 
including Colonel Dave Archino, Director, and Major Mike Grey, Chief, Operational 
Analysis Section. These IFFN Testbed personnel also participated in the MCES 
application to the IFFN Testbed. The following application is mainly taken from 
Major Gandee’s thesis, notes, and briefings along with Major Grey’s notes and 


conversations plus research of IFFN Test Force test plans. 


B. PROBLEM FORMULATION 

Utilizing the first’ MCES module, the MORS team and Major Gandee 
reformulated the [FFN problem which was a subset of the air defense C2 process 
problem. Within air defense c, the emphasis was on allocating multiple hostile 
targets to Weapons systems for engagement. Major Gandee understood that this air 
defense C* process description should consist of a complete set of battle management 
functions which were needed to direct the weapon systems to perform the air defense 
mission. Other issues considered by Major Gandee in the first module were the 
different evaluation levels and analysis objectives. Issues such as procedural control 
and centralized and decentralized control were also researched and reviewed. Figure 
4.1 lists the major actions taken by Major Gandee in applying MCES to the [FFN 
Testbed. 
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Figure 4.1 IFFN Application of MCES Module 1. 


The IFFN Testbed had focused its analvsis on specific concerns of Army, Air 
Force, NATO, and DOD decision makers regarding the role of identification as it 
contributes to the effectiveness of the air defense C* process. The major concern 
expressed by the IFFN Joint Testbed was the determination of how the programmed 
E- system and weapon systems would operate together. The IFFN mussion and its 
environment (friends, foe, neutral, weather) had alreadv been specified prior to Major 
Gandee’s MCES application. The friendly weapon systems Were limited to SANVIs and 
fighters with bevond visual range (BVR) munitions. A conventional threat scenario 
was already chosen by the IFFN Test Force so that stress on the ce system could be 
affected by varying traffic volume, ECM jamming to radars, communication jamming 
and varving weather conditions. [Ref. 1: p. 65] 

The final step addressed by Major Gandee in this module was the analysis 
objective. The overall air defense C2 analysis objective was reformulated bv the 
January 1985 MORS workshop from IFFN issues. The analysis objective [Ref. 2: p. 1] 


was to determine: 


How effective is the air defense C“ system in the central region.in Europe in 
providing decisionmakers the means to assess and employ air defense assets to 
meet overall mission objectives? 


47 


Major Gandee realized that identification can be affected by the presence of a physical 
entity or an asset like the airborne command post or by procedures such as those used 
for passing identification information. The IFFN analysis objective was eventually 
expanded by Major Gandee to determine: 


1. How effective is the C? process when the C2 structure and its attendant 
changes in tactics and procedures 1s varied. 


a Rok effective is the C¢ process when physical entities are added or lost. 
eiy2 pas 


The details of formulating the analysis objective involved interaction between the 


decisionmakers, operational users and analvsts. [Ref. 12: pp. 21-23] 


C. C* SYSTEM BOUNDING 

In the next module, the bounding of the Cc? system of interest was confirmed by 
\fajor Gandee from previous IFFN efforts. Figure 4.2 lists the condensed results of 
implementing Module 2 for the IFFN Testbed. Physical entities were identified and 
bounded and Figure 4.2 depicts the successful application of the “onion skin” idea. 
Alternative organizational structures were determined and hierarchal charts formulated. 
Again, much of this had been accomplished earlier at the IFFN Testbed but not by 
using specific {CES methodology. The application of this module confirmed that the 
IFFN C* bounding was sound. [Ref. 12: pp. 21-26] 


D. C* PROCESS DEFINITION 
1. Air Defense C* Process Functions 

In this module, Major Gandee defined the Cc? process functions of the 
distributed C2 air defense system. The air defense C2 process functions were 
determined to be: detect (D), track (1), identify (ID), assess threat (TA), assign 
weapon (WA), allocate weapon (AW), and weapons monitor and control (C). Figure 
4.3 depicts the air defense C2 process. These air defense C2 process functions Were 
mapped to the modified Lawson's Cc? process model to ensure that all C* functions 
had been considered and Figure 4.4 depicts that translation. [Ref. 12: pp. 32-37] 

These air defense functions represented what the air defense Ce system and 
weapon system are required to accomplish together to perform the mission. For the 
[FFN Testbed application, a process function was added to Lawson's generic Cc? loop 
and the plan function was eliminated. Lawson’s plan function did not correspond to a 
real-time activity during the IFFN execution phase, however, non-real time plans such 
as airspace control procedures (ACPs) and rules of engagement (ROE) are a part of 


Weapons assignment, allocation, and control. 
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Figure 4.2 IFFN Application of MCES Module 2. 
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Figure 4.4 Mapping of Air Defense C* Functions. 


2. Distributed C~ Process Interface 
Since the IFFN Testbed was dealing with a distributed ce system, the 
determination of C? process function boundaries was sometimes complex. Major 
Gandee discovered that in a distributed command and control system there are three 
distinct processes that will affect the overall performance of the ce system; intelligence, 
crosstell (coordination), and execution level C2 processess [INcl.ul2-Dp..56,57| 
a. XTELL Process 

A separate Crosstell (XTELL) process provides a way to share target 
information for the purpose of improving the overall picture of the environment and 
improving the accuracy of information. This is especially important for identification 
(ID) information in the air defense application where command centers are 
geographically dispersed. Individual command centers may develop definitive ID 
information which can be used by other command centers who have a tactical 
advantage or resources to engage the target. The XTELL process is accomplished 
through three functions of Crosstell (XTELL), Track Correlation (TC), and ID 
Conflict Resolution (IDC). The XTELL process is represented in Figure 4.5 with its 
(on process interface. 

ieee ilelumcianrol the NTE aprecess is the transfer and receipt of 
information via data link with some rules or filters. These rules specifv where 
information is to be sent and what information will be received. The Track Correlation 
(TC) function resolves location and track numbering disagreements in the on, system. 
The ID Conflict Resolution (IDC) function resolves conflicts that mav arise in the 
identification process between different C* nodes. At some nodes, this IDC function 1s 
a fusion process while at other nodes it 1s a decision process. 

Figure 4.6 represents the XTELL process in a lateral relationship. This 
lateral relationship represents adjoining units of the same level passing coordinating 
information between them. This information is then fused and correlated. A vertical 
or hierarchal XTELL relationship can also be present in distributed ce systems. The 
vertical NTELL process is similar to the lateral relationship except that now the 
coordinating information flows between the hierarchical related units. The fusion and 
correlation of the identity and track information may be different than that in the 
lateral relationship since the higher level unit will usually have more voice in resolving 
conflicts of information. The alternative C* systems have various configurations of 


these types of XTELL relations making some the configurations quite complex. 
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Figure 4.5 NATELL Process Functions and C* Process Interface. 


b. INTELL Process 

An Intelligence (INTELL) process aids decisionmakers throughout the Ce 
system in forming perceptions of enemy capabilities and intentions. The INTELL 
process is accomplished through four functions of Sense (S), Process (P), Intelligence 
Correlation (IC), and Assess (A). [Ref. 12: pp. 39-42] 

The function which collects data necessary to describe and forecast the 
environment is termed the Sense (S) function. The function that transforms data into 
information about the enemy forces’ disposition and actions is termed the Process (P) 
function. The Intelligence Correlation (IC) function correlates intelligence information 
with track and ID information. The Assess (A) function is performed when 
information is examined and patterns uncovered that indicate the actions or intentions 
of the enemy. The Assess function is also performed when patterns are utilized to 
forecast possible future changes in environment. Figure 4.7 graphically depicts the 
INTELL and C* process along with the XNTELL process. 
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Figure 4.6 Lateral NTELL Process. 


. ) 
c. Execution Level C- Process 


ice tee beande EEL processes suipport the Command and Control 


Process. The C- process can be viewed at a level which directly controls weapon 
systems and at a higher echelon which coordinates the efforts of Ge processes which 
direct the Weapon systems. Since the IFFN Testbed simulates the NATO air defense 
svstem which 1s geographically distributed, the Cc process included a netting of the 
separate command centers through the ANTELL process. The INTELL process will 
also be interfaced with some of the process functions. The interfacing of the XTELL, 
ies t ELL, and Ce processes together by communication links, protocols, operational 


procedures determined the overall C* architecture. Figure 4.8 lists the major actions 


completed in Module 3. [Ref. 12: pp. 44-46] 


INTEGRATION OF SYSTEM ELEMENTS AND FUNCTIONS 


Prior to developing measures, Gandee felt that a model or architecture that 


described the system was definitely needed. When Gandee attempted to establish an 
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Figure 4.8 IFFN Application of MCES Module 3. 
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architecture for the air defense C- svstem, he found that he needed a number of 
actions not listed in the then current \{CES methodology. The methodology for 
interfacing all the processes into an architecture was not covered completely in the 
original version of MCES. The developers of MCES adopted this idea of the 
integration of system elements and functions into an architecture to support the C- 
evaluation. 

1. Structures 

Information flow through the air defense Cc? process functions was used by 
Gandee to derive a natural hierarchical relationship between the individual C- 
functions in the form of an information flowchart. Later command organizations and 
equipment and communications alignment were also related to form a organizational 
Structunerchart. |Ref. 9: p.54] 

Major Gandee needed a methodology that documented this internal 
processing and described how the information is input to and output from the function. 
There are a number of methods available to formulate and describe the internal 
processing of C* functions. In this module, a specific software design technique, Data 
Flow-Oriented Design [Ref. 7: pp. 99-115] was used to integrate the system elements 
and functions of the C* svstem. Thus, this input.output relationship could form a 
description of the internal information flow between separate process functions as 
required to perform the mussion. The end result was a “structure” for a particular 
version of the C? system. The MCES definition of “structure” states that structure 
identifies the arrangement and interrelationships of phvsical entities, procedures, 
protocols, concepts of operation, and information patterns. [Ref. 12: p. 48] 

a. Data Flow-Oriented Design 

neenemincestep, each (on process function was examined and the data 
flowing through the function defined. A graphic representation of this process is 
termed a data flow diagram (DFD) and they describe the input,output relationships 
that exist between the C? functions. Figure 4.9 depicts a data flow diagram (DFD) for 
an “execution level” C2 process at a single command node. The DFD’s were also 
applied to interface the INTELL, XTELL, and Force processes with the Ce process in 
the distributed IFFN C2 system. Major Gandee described how that information flow 
linked those separate processes into an architecture of the complete C* or combat 
system. [Ref. 12: pp. 47,48] 
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Figure 4.9 Air Defense Single Node Data Flow Diagram. 


b. Transform Analysis 

Imsthe next step, the Cc? process as a Whole was reviewed and a transform 
analysis performed on the DFD to determine the Cc? process or transform center. 
Using this flow of information into and out of each function, a transform analvsis was 
conducted to determine the information transforming process center where information 
flowed in and were the information was transformed into an output in the form of 
control information. A C* function is analogous to a data flow transform. An 
example is shown in Figure 4.9 which depicts the Assess Threat function as the C 
process or transform center of the basic air defense process where the main perception 
is formed. Information flows in to formulate this perception and is termed the afferent 
branch. Information flows out in terms of decisions based on this perception and 
constitutes control flow and is termed the efferent branch. Structurally, this Assess 
Threat function subordinates the others and it was designated as the ce process center. 
A structure chart was derived from this transform analysis which shows the overall 
structural relationships between each Cc? process function. This process center was 
then used to establish the structural hierarchy between Cc process functions. This 
hierarchical relationship between C2 process functions was presented as a Structure 
chart. All these C? functions can be potentially performed in a single node or be 
distributed between different nodes. 

After the functional structure is formed, the people and their equipment 
can then be matched to that structure. Major Gandee’s gave an example of the battle 
commander performing the Assess Threat function supported by the identification 
officer. The commander also has subordinate weapon assignment officers who 
implement his decisions to attack the most important targets. Major Gandee found 
that the equipment consoles can be matched to this structure as capabilities to assign 
targets or control weapons can be implemented by configuring consoles to address 
their output in accordance with the specified “structure”. [Ref. 12: pp. 48-50] 

c. Null Process Concept 

Under some operational concepts, ce process functions can be distributed 
between command nodes such as Brigade and Battalion FDCs or between command 
nodes and weapon systems such as the CRC and fighter. The C2 process functions can 
be divided. Such arrangements are often temporary and unique to the particular 
version of the C? system. Major Gandee developed the concept of a null process to 


differentiate between the C* process functions when they are distributed. For example, 


57 


the Brigade FDC allocates to the Battalion FDC and the Battalion FDC allocates the 
Weapon system. Only one on process can direct a weapon system although its 
decisions mav be influenced by information coming from other Ce processes. Influence 
can come in the form of an indirect ID or priorities from a higher echelon. Each 
command node can potentially perform all C* functions to direct force actions in the 
environment. Figure 4.10 and Figure 4.11 depict the distribution of C? and force 
functions between a Battalion FDC and SAM battery for two differing operational 
concepts of centralized and decentralized control. A function can be null at a facility 
due to a physical limitation such as the null Detect function at the Battalion for lack of 
Organic radar or due to a redistribution of decision functions to reflect a different 
operational doctrine. 

With these techniques, the Cc? system architecture was changed to show 
relationships between “physical entities”, and “processes”, to produce a “Structure”. 
This structure was altered to reflect different operational concepts which form the 
different versions of the C? svstem. There should be a different structure for each 
alternative svstem. Figure 4.10 is a good example of the structure of a battalion 
emploving centralized control of its battery fire units and 1s different than the structure 
in Figure 4.11 of a Battalion emploving decentralized control of its batterv fire units. 
[In these illustrations, the null functions are not enclosed by a box. [Ref. 12: pp. 51-53] 

ad. Procedures 

Procedures are utilized in the internal processing within C* functions. For 
instance, some IFFN issues deal with ID value of air space control procedures. These 
rules or procedures are specified externally but used internally within the ID function 
to determine ID. These rules, when combined with other sources for [ID into some 
decision loop or algorithm, affect the internal “structure” of the ID function. If these 
procedures are taken away then the decision loop (internal structure) is changed. 
Major Gandee used a design technique which provides a module description that 
explodes each C? function to define the internal processing and the coupling to other 
C* or force functions. In this approach the functions were related to an appropriate 
physical entity prior to determining relevant measures of performance (MOP), 
measures of effectiveness (MOE), and measures of force effectiveness (MOFE). 
[Retaae: pp. 34,55] 
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Figure 4.11 Decentralized Control of a Battalion Fire Unit. 
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2. Architecture 

The MCES defines “architecture” as an description of a integrated set of 
systems Whose physical entities, structure and functions are coherently related. The 
architecture provides a representation which will eventually lead to the ability to 
measure the C~ system response and the effectiveness of directing forces to accomplish 
the mission. The integration of the system elements of man and machine with the 
process functions will eventually form an overall architecture that can be used by 
analvsts to evaluate the C- system. The final step is to formulate a overall Cc? system 
architecture that will incorporate all the different version structures internally. The last 
step in completing the architecture was to identify what physical entites performed the 
individual process functions and what connectivity linked the functions together. This 
established a single architecture represented as an overall structure chart. The final 
form of the architecture includes the process description and svstem elements 
performing the processes arranged in a structural framework. Additional modules to 
the structure chart provided documentation for equipment, personnel requirements, 
and connectivity in the necessary detail. 

The general C? architecture will remain unchanged but the structure variations 
would be represented by the different version’s unique information flow. An air 
defense example was illustrated by Major Gandee to describe how structures can differ 
internally in a Cc? system architecture. This illustration involved air defense operators 
located in front of consoles. Equipment consoles could be configured in various wavs 
to aid the operator in performing certain functions and allow the output to be 
transfered to other consoles. The operator would be aided in his ability to process 
information and communicate it through a machine structure that parallels an 
organizational structure. The general C* architecture would be the same but there 
would be unique structures utilizing the equipment and personnel differently. 
[Ref. 12: pp. 48-50] 

Major Gandee listed three advantages for utilizing an architectural 
representation of the Cc? system. Major Gandee found that the Cc? process can be 
broken down into separate functions and appropriate attributes defined moore 
svstematically than previous brute force or exhaustive listing methods. For example, 
the major Identify function attributes were relatively easy to determine as accuracy, 
timeliness and completeness. The second advantage of an architectural representation 


is the cababilitv of defining where a measure should be taken to measure a certain 
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function. Certain operational concepts have the same function performed at different 
nodes or levels. Therefore, measurements must take place where the function is being 
conducted. For example. the Allocate Weapons (AW) function is performed at the 
Battalion level in the Centralized Control mode, Figure 4.10, but 1S performedmamenas 
Fire Unit level, Figure 4.11 , in the Decentralized Control mode. The Battalionsiia 
monitors the -Allocate Weapons function activiues in the Decentralized Control mode. 
If an accurate architecture depicts the actual operations Of the c- system, these 
relauuionships are clearly delineated. A third verv important reason for architectural 
representation 1s its capability to graphically depict the c- svstem and weapon systems 
and highlight appropriate operational issues. [Ref. 9: pp. 54-56] 

Mfajor Gandee’s work did result in the addition of more objectives and 
ulumately additional measures to Test Series 2 as new relationships were uncovered. 
Figure 4.12 displays the major results for the implementation of \Mfodule 4 to the IFFN 
Tésroed: 
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Figure 4.12. IFFN Application of MCES Module 4. 


FL. SPECIFICATIONS OF MEASURES 
Major Gandee used a different approach than the IFFN Testbed when 


determining the measures needed to evaluate the competing versions of the air defense 


C- svstems. Instead of moving from issues to objectives to measures to answer those 
objectives, Mlajor Gandee examuned the information flow and bounded system elements 
to determine which measures were needed. Figure 4.13 graphically depicts Major 
Gandee’s approach for determining measures for the air defense Cc? system. In this 
figure, the shaded area represents the functional air defense ce system. The MOPs are 
measures of each function within the C* svstem. Each function is dependent on those 
functions preceding it, so the MOPs are conditional probability measures with 
additional range and timeliness measures not shown. \{OEs measure how well the Cc? 
system performed all its functions as a whole and is measured outside of the C? Sistem: 
1. MNICES Test Series 2 Issues 

Major Gandee illustrated some of the issues he considered concerning 
identification, centralized control, and network connectivity for Test Series 2. Focusing 

upon such issues lead to the differentiation among the alternative architectures. 
e [ssue I: Will centralized control at the Battalion manage the missile resources 
eelie by spreading the fire power more evenly over subordinate units and over 


e Issue 2: Under what. traffic volume conditions can centralized control be 
handled without degradation? 


e Issue 3: If the data links (XTELL process) carry information on which targets 
have been allocated for engagement, can SAMI batteries operate in a self- 
deconflicting manner conserving nussile resources? 

e [Issue 4: Given decentralized control and self-deconflicting doctrine, is a fullv 
connected data network required to prevent a single point failure due to the 
possible destruction of a Battalion Firing Unit? 


e Issue 5: Will the XTELL network supply the most complete ID to_the other 
28 batteries when their ID equipment becomes inoperable? [Ref. 9: pp. 
a 


2. MCES Measures 

Major Gandee’s first four issues for Test Series 2 were sensitive to structural 
changes. Major Gandee suggested possible efficiency and coordination measures which 
would reflect the probability of a target being engaged by more than one unit due to 
lack of coordination. The fifth issue of identification questioned whether a network 
could increase individual unit identification capabilities. This could be accomplished by 
supplving more complete [D information from other units which formulate the ID 
information. Major Gandee suggested that an ID accuracy measure could be used to 
compare the accuracv of an ID formulated at an organic unit and the ID information 
which passes over the network to other units as a svstem ID. Examples of generic 


MOEs are: timeliness, accuracy, survivability, capacity, and percent completion. 
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Figure 4.13 \{CES Approach for Determining Measures. 
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Other possible measures suggested by \Nfajor Gandee were fusing measures that 
measured the abilitv of units to accept and fuse svstem ID information with its own 
organic ID information to improve the 1D accuracy in time to use it effectively. Figure 
4.14 represents the major measures recommended by Major Gandee during the \ICES 


application of Module 5. [Ref. 12: pp. 71-76] 
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Figure 4.140 IFFN Application of MCES Module 5. 


G. MCES APPLICATION SUMMARY 

Major Gandee’s proposed application of \{CES to the 1FFN basically stopped at 
Module 5, Specification of Measures, due to time constraints and the delay of Test 
Series 2 execution. As a result, the MCES data generation and aggregation and 
interpretation modules were not evaluated for Test Series 2 by Major Gandee. The 
MCES methodology provided a evaluation methodology to assist the IFFN Test Force 
in its evaluation of the air defense C? problem. The MCES approach of systematically 
outlining the physical entities, structure, and Cc? process functions insured that all 
evaluation areas were covered. MCES has definitely helped in highlighting the 


functional measures that have been overlooked in previous C* evaluations. The 


distributed functions are required to characterize the coordination and intelligence 
sharing of distributed ony systems. There are different levels of these distributed 
functions and the C? structures can become verv complex. MCES’s distributed 
functions did assist in describing these complexities. The \ICES concept of using a 
model or architecture to establish a baseline with alternative structures to represent the 
competing Cc? svstems was very useful in developing measures to differentiate between 
them. Understanding the svstem has to take place before attempting to measure its 
utility and to uncover which variables are responsible. The ICES approach appeared 
to be detailed and complete and new relationships were uncovered by Major Gandee. 
MICES provided the IFFN ce analvsts with a theoretical framework for determining 
the utility of a Cc? svstem. The MCES apphcation did assist the IFFN Testbed. 
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V. ANALYSIS OF THE IFFN TESTBED AND MCES APPROACHES 


A. FEEDBACK TO THE DECISION-MAKER 

Major Grev, Chief, Operational Analvsis section, IFFN Joint Test Force, and the 
[FFN testbed director, Colonel Dave Archino, participated in the [ORS workshop 
and later incorporated some of the MCES ideas into its test plans. Major Gandee 
visited the IFFN Testbed, conducted his research, and made his MCES application to 
the IFFN Testbed with the full assistance of Major Grey. Major Gandee's proposed 
application of MCES to the IFFN basically stopped at Module 5, Specification of 
Measures, due to time constraints and the delay of Test Series 2 execution. As a 
result, the VICES data generation and aggregation and interpretation modules were not 
evaluated for Test Series 2 by Major Gandee. Due to even another delay in the 
execution of Test Series 2 until March 1987, the final results were not evaluated during 
the conduct of this thesis as was originally planned. 

Test Series 1 was planned and conducted without the aid of MCES. The early 
planning stages of Test Series 2 had been completed before the MCES application 
started. However, when new or better ideas were developed in the MCES application, 
some of these ideas were added to the Test Series 2 test plan. A strict comparison of 
the IFFN Testbed produced Test Series 1 results to Test Series 2 would not be valid 
due to the mixed participation in Test Series 2. In the evaluation of Test Series 2, it 
was somietimes difficult to determine exactly what part of the test plan was attributable 
to MCES or the I[FFN Testbed. In almost all cases, MCES at least confirmed earlier 
IFFN Testbed planning. In some cases, MCES provided new insights that were 


responsible for a better test plan. 


B. PROBLEM FORMULATION 

Although the problem formulation was completed earlier by the IFFN Test 
Force, MCES was used to verify that the correct steps were taken. As previously 
noted, the analysis objective was expanded by the MCES application. The problem 
formulation revolved around the air defense problem and not around how to build a 
credible testbed to evaluate competing air defense Cc? systems. The IFFN Testbed 
itself was a system that could be evaluated just as the IFFN Testbed was trying to 


evaluate air defense C7 systems. The IFFN Testbed was evaluated as part of its 
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certification effort. \feasures were formulated that would eventually determine if the 
[FFN Testbed produced valid and credible results. A comparison of the results was 
conducted between the values of the measures of target identification, allocation, and 
engagement from the IFFN Testbed simulation and actual Patriot livefire exercises. 
The building of the testbed itself was a background problem and will be covered in 


greater detail in the data generation discussion. 


C. C?SYSTEM BOUNDING 

The bounding of the C? system of interest was confirmed by Major Gandee from 
previous IFFN efforts. Physical entities were already identified and bounded. Much of 
this had been accomplished earlier at the IFFN Testbed but not by using the specific 
MICES methodology. The application of this module confirmed that the IFFN Cc? 
bounding was sound. The “onion skin” idea was a useful tool and was subsequently 


used by the IFFN Testbed to graphically display their bounding. 


D. C* PROCESS DEFINITION 

The IFFN Testbed originally identified three functions that were performed in 
the air defense process and those were: identification, target allocation, and target 
acquisition. The IFFN Testbed did conduct a functional analvsis of the air defense 
process though not in as much detail as the MCES functional analysis. Major Gandee 
and the MORS team defined the C” process functions of the distributed C? air defense 
svstem as: detect (D), track (T), identify (ID), assess threat (TA), assign weapon 
(WA), allocate weapon (AW), control (C) which were later adopted by the IFFN 
Testbed. 


E. SYSTEM ELEMENTS AND FUNCTIONS 

The IFFN Testbed did formulate the alternative C? systems that it wanted to 
test. Organizational and equipment charts were constructed as well as some 
information flow charts by the IFFN Test Force. The IFFN Testbed baseline criteria 
was basically a baseline architecture from which planned derivations would be tested. 
In this manner the IFFN Testbed accomplished an integration of system elements and 
functions but in less detail than Major Gandee’s application. The MCES application 
by Major Gandee was more thorough with his information flow charts. organizational 
charts, and structure charts. Alternative organizational structures were determined and 
hierarchal charts formulated by both the IFFN Testbed and MCES approaches, 
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however, \fajor Gandee’s structures were more detailed and complete. \fajor Gandee, 
through his use of null functions and adding or deleting physical entities, developed 
many different configurations for possible testing. The alternative configurations for 
centralized and autonomous (decentralized) control of the Patriot Fire Units was one 
of Major Gandee’s contributions. This concept was added to the IFFN Testbed’s 


analysis. 


F. SPECIFICATION OF MEASURES 
General categories of measures Were sought by the IFFN Testbed to derive 
values that would eventually lead to discriminating among the competing air defense 
c? systems. Early in the development of the IFFN Testbed, the analysts realized that 
they could deternune differences between the competing c? systems without precisely 
knowing which variables were responsible for the difference. As the development of 
IFFN Testbed progressed, the analysts knew they needed to determine which variables 
Were causing the difference. 
1. Deriving Measures 

When and where does the analyst derive measures in a Cc system? There 
MeTomtMenmcalerent dapproacnes considered by the IFFN Testbed as they derived 
measures to evaluate competing on Sustemis. Fee vkCES approach utilized the Cc? 
architecture and unique structures to derive their measures. The MCES methodology 
built a baseline C* architecture with alternative structures to derive where and when 
the measures should be determined. The IFFN Testbed approach utilized issues, 
objectives, and subobjectives to derive their measures and later built a _ baseline 
architecture to determine where to use the measures. The Institute for Defense 
Analysis (IDA) conducted extensive research in their attempt to determine measures 
for use by the IFFN Testbed [Ref. 13]. IDA also basically used a functional 
decomposition of the air defense C2 system to derive its measures. Aiphatec, Inc. 
Sewcloped a petri net model of the IFFN air defense Cc? svstem and then identified 
measures to determine the characteristics of each interconnection in the net [Ref. 14]. 
This massive effort resulted in over 200 measures for their five levels of the system. 

There were originally six objectives formulated for Test Series 2 that led to the 
IFFN Testbed measures. A new list of objectives was formulated after the MCES 
application and appeared in the next revised version of the Test Series 2 test plan and 
is listed below. Objectives 2. 6, and 7 were added to Test Series 2 after the MCES 


application uncovered the need for them. 
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e Objective | - Assess and contrast the performance of the PATRIOT battalion 
under centralized and decentralized control. 


e Objective 2 - Evaluate the contribution of adding Ce (Patriot Battalion FDC) to 
autonomous Patriot fire unit performance. 


> 


e Objective 3 - Assess the impact of changing and removing Airspace Control 
Procedures (ACPs) on the operational. performance of the centralized and 
decentralized battalion and autonomous fire unit. 


e Objective 4 - Determine the value_and impact of perfect ID and communication 
svstem performance on PATRIOT battalion performance. 


° perce 5 - Evaluate the impact of various changes in direct and indirect ID 
performance and interaction. 


e Objective 6 - Evaluate the contribution of Question and Answer (Q&A) [FF 
devices to Patriot Bn and FU performance. 


e Objective 7 - Evaluate the abilitv of indirect ID information to compensate for 
the loss of the direct O&A [FF device ar a earmomre. 


e Objective 8 - Assess the influence of the fire unit ID function on the ablity of 
the PATRIOT battalion to perform its functions. 


e Objective 9 - Identify and subjectively evaluate any PATRIOT operational 
deficiencies noted during testing. [Ref. [l: pp. 2.1,2.2] 


2. MOEs versus MOPs 

The terminology used by the IFFN Testbed for MOEs and MOPs is directly 
opposite of the MICES ternunology. MICES states that MOEs are measured outside of 
the C- system and that MOPs are measures of the functions within the Cc? system. 
MICES MOPs are used for the C? svstem measurements. Examples of MCES MOPs 
would be probability of detection and correct identification. Within the force 
boundarv, MCES Measures of Effectiveness (MOEs) are used for measuring the 
functions of the C? system. Examples of generic MCES MOEs are: timeliness. 
accuracy, survivabilitv, capacitv, and percent completion. MOFEs are used for the 
boundary measurements between the force and the environment. An MCES example 
might be the number of enemy aircraft destroyed prior to releasing their weapons. 

The IFFN Testbed termed the functional measures as \{OEs and their 
corresponding submeasures as MOPs. The IFFN Testbed MOEs for Test Series 2 
would measure the needed information for the C* evaluation objectives. MICES termed 
these tvpe of measures as Measures of Performance (MOP) since they were measures 
of the air defense functions. There is obvious disagreement in methodologies as to 
what to name the different set of measures. This was not an important detail and did 


not cause too much difhiculty. 


>. Functional Measures 


Major Gandee’s approach was to utilize the C* functions as the focal point 
for deriving measures. If this approach ts to be strictly followed, then each function 
would have at least one corresponding probability measure plus time and distance 
distribution measures. The measures of probabilities and time and distance ranges for 
each function should be the minimum measures used to measure these functions. 
\fajor Gandee proposed these measures and highlighted that all the functions should 


have corresponding measures. Table 4 lists the measures that should be included for 


the functions. 


TABLE 4 


MCES FUNCTIONS AND MEASURES 


FUNCTION 

DETECT (D) 

TRACK (T) 

IDENTIFY (ID) 

ASSESS THREAT (TA) 
ASSIGN WEAPON (WA) 
ALLOCATE WEAPON (AW) 


CONTROL (C) 


Most of these measures are listed in the test design plan for Test Series 2. \fajor Grev 
of the LFFN Testbed does state that the data for all these measures will be available 
but some were not considered for Test Series 2. Some of these functions will be 


measured indirectly and the functional measures may be incorporated in later tests if 
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the results of Test Series 2 reveals that they are needed. 
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4. Distributed Functions and Measures 

Major Gandee thought that the measures for the XTELL and INTELL 
processes must be used to effectively evaluate distributed Ce svstems. If this is indeed 
the case then each function should have at least one corresponding measure of 
performance. Examples of these are shown in Table 5. Most of these measures were 
used by the IFFN Testbed. The IFFN Testbed will use their measures of P(Pass), 
P(Res), P( Trans), and P(Amp) to measure the indirect ID information flow which will 
indirectly measure some of the coordination and intelligence functions. These IFFN 
mieasures and definitions are listed for review and comparison. 

e P(Pass) Probability that a passed ID 1s correct. 


e P(Res) Probabilitv that an ID conflict is resolved while the aircraft is still in 
the Weapon systems measurement volume. 


i eS Distribution_of times elapsed between receipt and retransmission of 
ID information by a C2 node. 


e P(Amp) Probability that an [ID includes track amplification information. 


TABLES 
DISTRIBUTED FUNCTIONS ANDIMOPS 


FUNCTION MEASURES 
CROSSTEEE Gti G15) P( Correct Fusion) 
Track Correlatiome( Le) Timeliness, accuracy 


{) Comics Timeliness, accuracy 
Resolution (IDR) 


INTELLIGENCE ( INTEEE) Bt eee Sod an given 
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IID was not used) 
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5. Operational versus Design/ Qualitative Measures 
While most evaluations of C? svstems center around design qualitative 

measures such as flexibility, surviveability, availability, etc., the IFFN Testbed used an 
approach much like the MICES methodology in that the functions of the system were 
first studied before equipment and personnel problems were considered. These 
measures are sometimes refered to as operational measures. Examples of operational 
measures for distributed Cc? systems are: 

e CAPACITY 

Peace uRACY 

See Sen LL VIE 

e AVAILABILITY 

oe el Py 
The IFFN Testbed did not continue their evaluation to include the design’qualitative 
measures that are definitely needed to determine the utility of a particular competing 
Cc? system. Major Grey did state that design type measures would be incorporated in 
later tests and that the data needed for these types of measures was readilv available 
even for Test Series 2 if the need arose or if a higher authority requested that 
information. Examples of design and quality measures are: 

eer Gale NCY 

e RELIABILITY 

e SURVIVABILITY 

e USEABILITY 

eeeORREG LT NESS 

e MAINTAINABLITY 

 VERIFIBILITY 

e EXPANDABILITY 

etre SB TY 

e INTEROPERABILITY 

e PORTABILITY 

e REUSABILITY 

e ROBUSTNESS 

e EVOLVABILITY 

Another analvst, Leslie Golliday, listed a set of generic air defense measures, 

Table 6 [Ref. 15: p. 788], which are similar to what the MCES application had 
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determined. These measures are more general but the specific measures could be 
derived from the general measures. Again, the list included function measures as well 


as design qualitative measures. 


TABLE 6 
C2 MEASURES FOR ATR DEFENSE 


MOP 


Alerting 
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Cueing_. 
Capability 


Weapons Control 
Information 
Capability 


Airspace 
Management 
Capability 
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avoidance of engagement of 
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MOES 
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EE Slee 
USESS 1 baa, 
SURVIVAB Ea 





6. Resource Conservation and Reallocation Measures 
Major Gandee suggested possible efficiency and coordination measures which 
would reflect the probability of a target being engaged by more than one unit due to 
lack of coordination. A network may increase an individual unit’s identification 


capabilities by supplying more complete ID information from other units which 


formulate the ID information. Major Gandee suggested that an ID accuracy measure 
could be used to compare the accuracy of an ID formulated at an organic unit and the 
ID information which passes over the network to other units as a system ID. Other 
possible measures suggested by Major Gandee were fusing measures that measured the 
ability of units to accept and fuse system ID information with its own organic ID 
information to improve the ID accuracy in time to use it effectively. However, the 
suggested measures of weapon allocation efficiency, unit ID coordination, [D accuracy, 
and ID fusing ability were not used by the IFFN Testbed. Some of these attributes 
could be measured indirectly using some of the other IFFN Testbed measures such as 
timeliness and the probability of correctly identifying an object to measure ID 
information fusing ability. 

Determining which competing C* consumes less missile resources is very 
important considering the cost and shortage of modern missiles. Also, in a dynamic 
environment, there will be instances when missile resources will have to be reallocated 
due to prior destruction of the target, previous allocation to another weapon, higher 
priority targets in the area or the targets flies out of range. Measures of efficiency and 
reallocation ability seemed to be crucial differences in the competing ce systems. 
Again, Major Gandee suggested such measures but they were not all incorporated into 
the current Test Series 2 plan. Research by Alphatec, Inc. with Petri nets also 


suggested reallocation measures for the IFFN Testbed [Ref. 14]. 


G. DATA GENERATION AND TESTBED DESIGN 

Major Gandee’s proposed application of MCES to the IFFN basically stopped at 
Module 5, Specification of Measures due to time constraints and the delay of Test 
Series 2 execution. As a result, the MCES data generation and aggregation and 
interpretation modules were not evaluated for Test Series 2 by Major Gandee. 

The current MCES does incorporate a experimental design methodology for 
building testbeds to generate data. A methodology, Systems Effectiveness Analysis 
(SEA), was introduced by A. H. Levis and P. Derskin [Ref. 16] for evaluating large 
scale systems such as testbeds and ultimately integrated into the MCES methodology. 
To produce valid data, a testbed must be credible and SEA was developed to assist in 
validating testbeds. SEA was also developed to determine the munimum number of 
experiments needed to evaluate the system and to formulate a optimal sequence of 


improvements areas for the competing configurations. Once the testbed was 
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determined to be credible, experiments could be run that would determine the optimal 
effectiveness of each alternative C* system. 

One of Dr. Levis’ students, Phillipe Martain, demonstrated how SEA could be 
applied to the IFFN Testbed. First in the SEA process was the determination of the 
smallest number of experiments that were needed to be run to evaluate the 
effectiveness of the testbed system. A simplified mathematical model utilizing 
Lanchester tvpe combat models was developed to determine the minimum number of 
experiments required to evaluate the testbed. The testbed experimental results and the 
mathematical model results could also be compared to insure similar results. In a 
second phase, a system planning procedure was used to select the best evolution path 
for the testbed configurations from a fixed set of improvements. SEA can be then used 
in the last ICES module of Data Aggregation and Interpretation to make adjustments 


to the testbed exmetiments. |. eres 


H. OVERLAPPING OF PROCEDURES AND TOOLS 

The IFFN Testbed has been working on its air air defense Cc? problem for a 
number of vears and through its iterative process evolved to a solution that was close 
to the MCES solution. The IFFN Testbed started without the aid of MCES and some 
of the applied MICES methodology overlapped with the previous methods used by the 
[IFFN Joint Test Force. However, in most cases both methodologies resulted in the 
same general results. The IFFN Testbed did make changes after the MCES 
application, but it is not clear if these changes will have a major impact on Test Series 
2 SINCE Ine Test nas netbeenscompleted: 

MCES does integrate and imbed current evaluation tools. MCES does not 
preclude these tools and in fact uses them to obtain a better solution to the problem. 
Both the IFFN and MCES methods came to the same general conclusions, however 
the MCES approach appeared to be more complete. The MCES methodology 
attempts to standardize the analysis by providing a structured template to assist the 
analysts in their evaluations. 

|. Problem Formulation and Bounding 

The problem formulation and bounding of the C2 system were almost 


identical. Most svstem analysis methodologies start out in this same manner. 
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2. Functional Analysis 

Other system analysis approaches use the basic input, process, and output 
approach to describing the Cc system. Functional and process analysis are being used 
often in the software engineering environment. These approaches are similar to C. 
West Churchman’s svstem process of input, process, and output as explained by 
Schoderbek, et al. [Ref. 18: pp. 8-29]. It is interesting to note that major methodology 
revisions occured when analysts attempted to automate svstems because they needed to 
precisely describe and recreate the functioning of the manual system. 

3. Model or Architecture Building 

The system analysis approach of utilizing the functions of the svstem to build 
a svstems model has been used in a number of previous evaluations. Other researchers 
have added methods of modeling that can be integrated into the MCES methodology. 
A specific example already referenced is Systems Effectiveness Analysis (SEA). Dr. 
Levis has conducted stimulating research in this bottom-up approach and has 
integrated it into the MCES methodology [Ref. 19]. The SEA methodology insures 
that the simulation can simulate the whole range or limits of the interactions between 
the variables instead of a smaller subset of the interaction range or limit. This can be 
accomplished by taking established measures and and determining their minimum and 
maximum ranges. The other quantitative methods of SEA can also be applied to the 
ieee bestbed.- 

Petri Nets have been used by researchers and analysts to mathematically 
model the different structures of information flow in the C? architecture in the form of 
off and on states which can be used to describe a C? architecture with unique 
structures. Alphatec, Inc. [Ref. 14] conducted a study of the IFFN Testbed and 
constructed a number of different level petri nets to determine what measures were 
needed to measure what kind and how much information flowed between all the nodes. 
Basically, each connection between the nodes was an opportunity to measure 
information flow. 

4. IFFN and MCES Measure Specification 

The IFFN Test Force used its issues to formulate their original measures. 
However, there were no major deficiencies in the test design of Test Series | which was 
completed prior to the MCES application. There was prior research conducted by the 
Institute for Defense Analvsis [Ref. 13] and Alphatec [Ref. 14] concerning the 
evaluation of the IFFN competing Cc? systems and they confirmed the measures 


derived by the MCES application. 
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i COMPARISON SUMMARY 

The [FFN Testbed has already taken advantage of the good ideas generated by 
the MICES application and the Test Series 2 plan has incorporated some of the MCES 
concepts. Although each methodology used different and similar tools to evaluate the 
competing or, systems, both approaches came to the same general conclusions. The 
question of Whether the amount of time needed to document all possible interactions in 
the MCES is really needed is still unanswered. However after utilizing the MCES 
approach of analyzing the phvsical entites, organizational structure, and C? functions 
in a systematic methodology, the IFFN Test Force discovered a number of important 
measures that they had not focussed on earlier in the test design process for Test Series 
2. Only with more testing and comparisons will the true value of the MCES approach 
to the I[FFN Testbed be known. Conclusions and recommendations concerning both 
the IFFN Testbed and MCES are included in Chapter VI of this thesis. 
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VI. CONCLUSIONS AND RECOMMENDATIONS 


A. IFFN CONCLUSIONS 

It is clearly evident that the IFFN Testbed has made some progress in solving 
some of the IFFN air defense problems. The problem ts definitely complex and the 
IFFN Testbed has understandably committed large amount of resources to the 
problem. The Testbed is a good concept for an experimental design to test alternative 
ce systems since all of the alternative systems can not be tested using actual 
equipment due to resource constraints or present Cc? configuration limitations. The 
IFFN Test Force was careful to insure that the testbed was credible. Only one test 
series has been completed with Test Series 2 to begin in March 1987. The IFFN Joint 
Testbed has provided an excellent opportunity to test and refine MCES. The [FFN 
Testbed is still valuable as a suitable data generator for further evaluatation and 
refinement of MCES. 

The IFFN Testbed has already taken advantage of the good results generated by 
the MCES application and the Test Series 2 plan has incorporated some of the MCES 
ideas. After utilizing the MCES approach of analyzing the physical entites, 
organizational structure, and C? functions ina systematic methodology, the IFFN Test 
Force discovered a number of important measures thev had not focussed on earlier in 
the test design process for Test Series 2. The test design issues and measures were 
modified to accommodate the newly found distributed relationships between C* nodes 


that were originally not formulated by the IFFN Joint Test Force. 


B. ITFFN RECOMMENDATIONS 

An additional measure approach would be to utilize both operational and 
equipment design, quality measures. The functional measures are operational measures 
and the design;quality measures are more machine and resource oriented. The IFFN 
Testbed seems to have focused its measures on operational or functional measures and 
has not taken full advantage of available and possibly critical design qualitative 
measures such as resource efficiency. 

The IFFN Testbed should consider Major Gandee’s recommendations on the 
additional measures of performance and effectiveness, particularly the measures of 


coordination and resource allocation. Resource allocation, connectivity, availability, 
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surviveability, sustainability, and flexibilitv data appears to be available from the data 
collection points in the simulations. The measures for the XNTELL and INTELL 


processes must be used to effectively evaluate distributed ce systems. 


C. MCES APPLICATION CONCLUSIONS 

The [F FN Testbed has been working on its air defense ce problem for a number 
of vears and through its iterative process evolved to a solution that was close to the 
same results. There is a learning curve associated with applying any new methodology 
Which was quite evident in this IFFN application. During the MCES application to 
the [FFN Testbed, there was overlap of the evaluation tools used by the IFFN Test 
Force. Some of the tools used prior to the MCES application to the IFFN Testbed 
Were included in the MCES methodology. MCES does not preclude these tools and in 
fact uses them to aid in the evaluation to obtain the best results. This MCES 
application was a good start in the evolving of a generic standard Cc? svstem evaluation 
method. The MCES approach of systematically outlining the physical entities, 
structure, and C? process functions insured that all areas were covered. Major 
Gandee’s proposed application was used bv the [FFN Operational Analvsis section to 
better understand their air defense C? problem. The MCES approach seemed to be 
more detailed and complete. New relationships were uncovered by Major Gandee 
resulting in the addition of new issues and measures. MCES has definitely helped in 
highlighting the functional measures that have been overlooked in previous Ce 
evaluations. NICES did assist the [FFN Testbed. 

1. Integration of System Elements and Functions Module 

\[ajor Gandee uncovered the need for the additional module 4, Integration of 

Svstem Elements and Functions, that was not originally conceived as a part of the 
\ICES modules. A model or architecture was needed to establish a baseline with 
alternative structures to represent the competing ce svstems. The system must be 
understood before attempting to measure its utility and uncover the variables are 
responsible for significant differences. By actually using MCES, some problems 
surfaced which ultimately resulted in refinements to the MCES methodology. 

2. Distributed C* Process Functions 

The distributed functions are required to characterize the coordination and 

intelligence sharing of distributed or, svstems. There are different levels of these 
distributed functions and the C? structures can become very complex. \ICES's 


distributed functions assist in describing these complexities. 
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3. Solid Evaluation Tool 
\ICES appears to be a viable C* evaluation template of current and evolving 
tools based on solid C? theorv. MCES has provided the ony analyst community with a 
theoretical framework for determining the utility of a Cc system. 
4. MCES Application to Additional IFFN Testbed Test Series 
The IFFN Testbed did revise their Test Series 2 plan to incorporate most of 
Major Gandee’s results of applying MCES. Major Grey has also utilized some of the 
concepts of MCES to formulate the design plan for Test Series 3. As more test series 


are completed, a more through evaluation of MCES can be made. 


D. MCES RECOMMENDATIONS 
1. Continued M(CES Application to the IFFN Testbed 
The application of MCES to the IFFN Testbed should be continued on Test 
Series 2 and following tests. Due to time constraints and the delav of the Test Series 2 
execution, the final results were not available for analysis in this thesis as was originally 
envisioned. The areas of actual data generation and aggregation of measures and 
interpretation for Test Series 2 still could be a valuable thesis topic for a follow-on 
project. This follow-on project could document and evaluate the success of the 
previous implementation of MCES and observe new implementations. A comparison 
of Test Series 1 with Test Series 2 results might reveal the differences in the approach 
and the results with the caveat that Test Series 2 was a slightly different simulation 
tian lest’ Series |. 
2. Further MCES Testing and Refinement 
More beginning to end Cc? applications of MCES should be conducted to 
continue the evolution of the MCES methodology. 
3. Integration of More C* Evaluation Tools 
MCES does integrate a number of tools and could still integrate more while 
maintaining its solid theoretical base. The MCES approach is a top-down systems 
approach with certain advantages and disadvantages. Utilizing Dr. Levis’ experimental 
design and bottom-up approach, Systems Effectiveness Analysis (SEA), would benefit 
the MCES toolbox. Petri nets show promise of being a good analyst tool for 
evaluating information flow. Other system evaluation methods should also researched 


for addition to the MCES methodology. 
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4. Standard NICES Terminology and Definitions 


VICES developers must decide on standard terminology and definitions to 
avoid confusion. Although MICES should be robust and flexible to analysts, a firm 
evaluation theory must be presented in simple, standard terminology. 

5. Education and Dissemination 

There is a learning curve associated with any new or revised methodology 
which was quite evident in this IFFN application. After a standard terminology is 
defined, MICES should be announced and advertised to the Cc? analyst and decision 
maker community because of its sound theoretical background. More disclosure of 
MCES is definitely needed. Since MCES does incorporate a number of known tools, 


MICES should be advertised as a structured approach to utilize C* evaluation methods 
and tools. 
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